Feature construction using explanations of individual predictions

Feature construction can contribute to comprehensibility and performance of machine learning models. Unfortunately, it usually requires exhaustive search in the attribute space or time-consuming human involvement to generate meaningful features. We propose a novel heuristic approach for reducing the search space based on aggregation of instance-based explanations of predictive models. The proposed Explainable Feature Construction (EFC) methodology identifies groups of co-occurring attributes exposed by popular explanation methods, such as IME and SHAP. We empirically show that reducing the search to these groups significantly reduces the time of feature construction using logical, relational, Cartesian, numerical, and threshold num-of-N and X-of-N constructive operators. An analysis on 10 transparent synthetic datasets shows that EFC effectively identifies informative groups of attributes and constructs relevant features. Using 30 real-world classification datasets, we show significant improvements in classification accuracy for several classifiers and demonstrate the feasibility of the proposed feature construction even for large datasets. Finally, EFC generated interpretable features on a real-world problem from the financial industry, which were confirmed by a domain expert.Comment: 54 pages, 10 figures, 22 table

Gender, competition and performance: evidence from chess players

This paper studies gender differences in performance in a male‐dominated competitive environment chess tournaments. We find that the gender composition of chess games affects the behaviors of both men and women in ways that worsen the outcomes for women. Using a unique measure of within‐game quality of play, we show that women make more mistakes when playing against men. Men, however, play equally well against male and female opponents. We also find that men persist longer before losing to women. Our results shed some light on the behavioral changes that lead to differential outcomes when the gender composition of competitions varies

Search and Knowledge for Human and Machine Problem Solving

In Artificial Intelligence (AI), there exist formalised approaches and algorithms for general problem solving. These approaches address problems that require combinatorial search among alternatives, such as planning, scheduling, or playing of games like chess. In these approaches, problems are typically represented by various kinds of graphs, and problem solving corresponds to searching such graphs. Due to their combinatorial complexity, these problems are solved by heuristic search methods where problem-specific heuristics represent the solver’s knowledge about a concrete problem domain. Thus such computer-based approaches to problem solving roughly consist of two components: search among alternatives, and problem-specific knowledge. Computer methods of heuristic search are also a good model of human problem solving. In human problem solving, however, these two components take very different dimensions compared with machine problem solving. A human expert typically uses much richer domain-specific knowledge whereas the computer has the advantage of incomparably faster search compared to the human. The thesis presents some novel aspects on the comparison and combination of search and knowledge in human and machine problem solving, in particular with respect to possibilities of developing heuristic-search methods for evaluating and improving problem-solving performance. Among others, the following scientific questions are addressed. How can a computer be used to assess a human’s problemsolving performance? How can a machine problem solving model be used to assess the difficulty of a given set of problems for a human? How can machine problem solving be used in tutoring, for teaching a human to solve problems in a given problem domain? How can knowledge represented in a form suitable for the computer, be transformed into a form that can be understood and used by a human? In this thesis we explore these questions in the framework of human and computer game playing, and use the game of chess as the experimental domain. In Part I of the thesis, “Search and Knowledge for Estimating Human Problem Solving Performance,” we demonstrate that heuristic-search based programs can be useful estimators of human problem-solving performance. We introduced a novel method, based on computer heuristic search, for evaluating problem-solving performance in chess (with possible extensions to other games), and provided an analysis of appropriateness of this method. Experimental results and theoretical explanations were provided to show that, in order to obtain a sensible ranking of the chess players using our method, it is not necessary to use a computer that is stronger than the chessplayers themselves. We also designed a heuristic-search based method for assessing the average difficulty of a given set of chess positions (problem situations). In Part II, “Search and Knowledge for Improving Human Problem Solving Performance,” we presented a novel, heuristic-search based approach to automated generation of human understandable commenting of decisions in chess. We also demonstrated a novel approach to the formalization of complex patterns for the purpose of annotating chess games using computers. Finally, we introduced a procedure for semi-automatic synthesis of knowledge suitable for teaching how to solve problems in a given domain. We verified appropriateness of this procedure in a case study where we applied it to obtain human-understandable textbook instructions for teaching a difficult chess endgame. Part III, “On The Nature of Heuristic Search in Computer Game Playing,” aims at improving the understanding of properties of heuristic search and consequences of the interaction between search and knowledge that typically occurs in both human and machine problem solving. Monotonicity property of heuristic evaluation functions for games was revisited. Namely, that backed-up values of the nodes in the search space have to tend to approach monotonically to the terminal values of the problem state space with the depth of search. We pointed out that backed-up heuristic values therefore do not approximate some unknown “true” or “ideal” heuristic values with increasing depth of search, in contrast to what is generally assumed in the literature. We also discussed some of possible impacts of this property on the theory of game playing, and pointed out that heuristic evaluations obtained by searching to different search depths are not directly comparable, in contrast to what is generally assumed both in literature and in practical applications. Finally, we studied experimentally factors which influence the behavior of diminishing returns with increased search. Empirical proof was provided that the rate of changed decisions that arise from search to different depths depends on (1) the quality of knowledge in evaluation functions, and (2) the value of a node in the search space

Računalniška analiza svetovnih šahovskih prvakov

Estimates about who was the strongest World Chess Champion of all times , are based primarily on the analyses of their games done by chess grandmasters and these are often subjective. The emergance of better and better chess programs, which can nowadazs already cope with the best chess players in the worls and even surpass them in certain elements, help gaining more objective answers to this question. Regardless of that, in past researches computers were mainly used for processing statistical data of the chess games' results. I had a different approach at evaluating World Chess Champions; I was mainly interested in the quality of their game, which I was evaluating with the help of computer analysis of individual moves. For this purpose I used a computer chess program, Crafty, which is renowned for being the most powerful open source chess program in the world. I altered the program so that it would not have a time limit at evaluating indvidual moves and set a maximum fixed depth of processing the search tree. By doing so, I enabled it to function on any compueter regardless of its speed, without that having to influence the results, while at the same time forcing it to dedicate more time to more difficult positions. I compared the Wold Champions based on different criteria, such as deviations of played moves versus the moves chosen by a computer , estimating errors and blunders, and the percantage of played out best moves. In addition to evaluating played moves, I also estimated the difficulty of positions with which the plazers were faced, and deviations of best two suggested moves in them. Based on these estimates I was ascertaining the expected game quality of World Champions under balanced conditions, which represents an attempt to bring the champions to a common denominator while evaluating regardless of their different game styles.l The results of the analyses also provided me with an overview of ups and downs of game quality in the player' careers and ascertaining their form in indvidual duels. I proved Crafty's credability as an evaluator as well as the correctness of the used working methods, by finding the players' errors coincided with actual game results and by confirming the dependence of errors on the complexity of the positions. So who was the best world Champion of all times? The winner according to the main criteria, where we measured average deviations between evaluations of played moves and best evaluated moves according to the computer, is Jose Raul Capablanca, who was the World champion during 1921 and 1927. He was also on top according to all other criteria, where we measured game quality, and was only beaten in one criterion, agme quality in different game styles, by the youngest World Champion, Vladimir Kramnik. Both of them distinctly deviated from the rest

Računalniška analiza svetovnih šahovskih prvakov

Estimates about who was the strongest World Chess Champion of all times , are based primarily on the analyses of their games done by chess grandmasters and these are often subjective. The emergance of better and better chess programs, which can nowadazs already cope with the best chess players in the worls and even surpass them in certain elements, help gaining more objective answers to this question. Regardless of that, in past researches computers were mainly used for processing statistical data of the chess games' results. I had a different approach at evaluating World Chess Champions; I was mainly interested in the quality of their game, which I was evaluating with the help of computer analysis of individual moves. For this purpose I used a computer chess program, Crafty, which is renowned for being the most powerful open source chess program in the world. I altered the program so that it would not have a time limit at evaluating indvidual moves and set a maximum fixed depth of processing the search tree. By doing so, I enabled it to function on any compueter regardless of its speed, without that having to influence the results, while at the same time forcing it to dedicate more time to more difficult positions. I compared the Wold Champions based on different criteria, such as deviations of played moves versus the moves chosen by a computer , estimating errors and blunders, and the percantage of played out best moves. In addition to evaluating played moves, I also estimated the difficulty of positions with which the plazers were faced, and deviations of best two suggested moves in them. Based on these estimates I was ascertaining the expected game quality of World Champions under balanced conditions, which represents an attempt to bring the champions to a common denominator while evaluating regardless of their different game styles.l The results of the analyses also provided me with an overview of ups and downs of game quality in the player' careers and ascertaining their form in indvidual duels. I proved Crafty's credability as an evaluator as well as the correctness of the used working methods, by finding the players' errors coincided with actual game results and by confirming the dependence of errors on the complexity of the positions. So who was the best world Champion of all times? The winner according to the main criteria, where we measured average deviations between evaluations of played moves and best evaluated moves according to the computer, is Jose Raul Capablanca, who was the World champion during 1921 and 1927. He was also on top according to all other criteria, where we measured game quality, and was only beaten in one criterion, agme quality in different game styles, by the youngest World Champion, Vladimir Kramnik. Both of them distinctly deviated from the rest